Image pickup apparatus, image pickup method, and program

ABSTRACT

An image pickup apparatus includes: a polarization removing element layer including a plurality of polarization removing elements, each of the plurality of polarization removing elements being configured to polarize an incident light to thereby obtain light having a polarization axis in a predetermined direction and to transmit the polarized light; a polarizer layer provided in front of the polarization removing element layer, the polarizer layer being configured to twist polarization axes of light having a plurality of polarization axes by predetermined angles, respectively, and to transmit the light; an inclination detecting device configured to detect inclination of the incident light to an optical axis; a controller configured to control and drive the polarizer layer such that the polarizer layer twists the polarization axis of light depending on the inclination detected by the inclination detecting device and transmits the light; and an image pickup device configured to detect the light.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a Continuation of U.S. application Ser. No.14/328,113, filed Jul. 10, 2014, which claims the benefit of JapanesePriority Patent Application JP 2013-154456 filed Jul. 25, 2013, theentire contents of which are incorporated herein by reference.

BACKGROUND

The present disclosure relates to an image pickup apparatus, an imagepickup method, and a program. Specifically, the present disclosurerelates to an image pickup apparatus, an image pickup method, and aprogram capable of removing a polarization component appropriately evenif an image-pickup posture is changed, and capable of taking an imagefrom which a polarization component is removed appropriately.

In the past, there is used the following technology, to which atechnology of extracting a polarization component is applied. Apolarization filter is mechanically inserted in/removed from an opticalpath of a camera manually or automatically. The polarization filter isconfigured to absorb or reflect a polarization component to be removed.As a result, a desired polarization component is extracted. A takenimage includes the extracted desired polarization component.

Further, the following technology is also used. That is, a physicalpolarizer is provided. A polarizer is rotated electrically in order totake an image including orthogonal polarization components.

However, according to the technology of extracting a polarizationcomponent mechanically by inserting/removing a polarization filter or byrotating a polarizer, the size of the apparatus structure is increased.In view of this, there is proposed a technology of extracting apolarization component electrically.

There is proposed the following technology, to which a technology ofelectrically extracting a polarization component is applied. That is, apolarization control device uses a GH (Guest-Host) liquid crystal (alsoreferred to as dichroism liquid crystal). The GH liquid crystal onlyabsorbs a predetermined polarization component. The polarization controldevice controls transmittance. As a result, an unpolarized natural lightis modulated, and an image is taken (see Japanese Patent ApplicationLaid-open No. 2005-173493). In the polarization control device, two GHliquid crystal layers are arranged as follows. That is, a polarizationaxis of one polarization component is orthogonal to a polarization axisof the other polarization component. The two GH liquid crystals areturned on/off alternately. As a result, the image pickup devicetransmits only one polarization component out of the light, which entersthe image pickup device from the outside-light incident side.

Further, there is also proposed the following technology employing theGH liquid crystal. That is, one GH liquid crystal layer functions as aCF (Color Filter) of one polarization component. As a result, a ND(Neutral Density) filter and a device configured to detect arbitrarypolarization are realized (see Japanese Patent Application Laid-open No.H05-048060).

SUMMARY

Meanwhile, let's say that the above-mentioned technology of JapanesePatent Application Laid-open No. 2005-173493 is applied to a camera. Inthis case, when an image having an arbitrary polarization component isto be taken, if a camera is inclined, the polarization orientation of anobject (objective world) is deviated from the polarization orientationof the camera. As a result, the predetermined polarization component maynot be absorbed well. Extraction of light having the only predeterminedpolarization component may be failed.

More specifically, as shown in the upper part of FIG. 1, the camera 11is arranged in the horizontal direction appropriately. In thisimage-pickup posture (camera orientation is appropriate), the horizontalpolarization component is to be transmitted. The GH liquid crystal GH1is configured to absorb the perpendicular polarization component. The GHliquid crystal GH1 is turned off (i.e., absorbs the perpendicularpolarization component). The GH liquid crystal GH2 is configured toabsorb the horizontal polarization component. The GH liquid crystal GH2is turned on (i.e., does not absorb the horizontal polarizationcomponent). As a result, both the GH liquid crystals GH1 and GH2 do notabsorb the polarization component of the incident light from the objectin the horizontal direction. As a result, the horizontal polarizationcomponent passes through the GH liquid crystals GH1 and GH2. As aresult, the image Ph1 having the horizontal polarization component isextracted as it is as the image Ph11. The image Ph11 is a transmittedlight having the horizontal polarization component.

Meanwhile, the GH liquid crystal GH1 absorbs the polarization componentof the incident light from the object in the perpendicular direction. Asa result, the polarization component may not pass through the GH liquidcrystal GH1. The image Pv1 has the perpendicular polarization component.The image Pv1 is extracted as the image Pv11. The image Pv11 hasdarkened transmitted light.

As a result, the image Pv11 having the perpendicular polarizationcomponent is darkened. Substantially, the image Ph11 having thehorizontal polarization component is only extracted.

However, as shown in the lower part of FIG. 1, for example, the camera11′ is inclined at 45 degrees to the horizontal direction. In thisimage-pickup posture (camera orientation is inclined at 45 degrees), theGH liquid crystal GH1, which is configured to absorb the perpendicularpolarization component, is turned off (i.e., absorbs the perpendicularpolarization component). The GH liquid crystal GH2, which is configuredto absorb the horizontal polarization component, is turned on (i.e.,does not absorb the horizontal polarization component). Even in thiscase, because both the GH liquid crystals GH1 and GH2 are inclined at 45degrees, the GH liquid crystal GH1 absorbs a part of the horizontalpolarization component out of the incident light from the object.However, a part of the horizontal polarization component passes throughthe GH liquid crystal GH1. As a result, the image Ph1, which includeslight having amplitude in the horizontal direction, is extracted as theimage Ph1′. The image Ph11′ includes transmitted light, which isobtained by reducing a part of the polarization component by the GHliquid crystal GH1.

Meanwhile, the GH liquid crystal GH1 also absorbs a part of light havingamplitude in the perpendicular direction out of the incident light fromthe object. However, a part of light having amplitude in theperpendicular direction passes through the GH liquid crystal GH1. As aresult, the image Pv1 including the perpendicular polarization componentis extracted as the image Pv11′. The image Pv11′ includes transmittedlight, which is obtained by reducing a part of the polarizationcomponent by the GH liquid crystal GH1, similar to the image Pf11′.

As a result, the dark image Pv11′ including the perpendicularpolarization component is extracted. The dark image Ph11′ including theamplitude component in the horizontal direction is extracted. As aresult, a dark image is merely obtained as a whole (i.e., darkened imageis merely obtained). It may not be able to extract an image including apredetermined polarization component.

Further, the following technique may be employed. That is, apolarization filter is mechanically inserted in/removed from an opticalpath of a camera. The polarization filter is configured to absorb orreflect a polarization component to be removed. However, according tothis structure, it is necessary to operate two orthogonal polarizationfilters at high speed. In this case, there is a limit of speed whenpolarization components are obtained.

Further, the size and weight may be increased because it is necessary toprovide a mechanical mechanism. Further, when a camera is inclined, apolarization component to be reduced will not be reduced sufficiently.In addition, the signal level of the polarization component to bedetected simultaneously will be decreased.

Further, when a camera is inclined, it is necessary to prepare apolarization filter, which is configured to rotate the polarizationdirection depending on the inclination of the camera, and toinsert/remove the polarization filter in order to detect thepolarization component with a high degree of accuracy. Because of this,it is necessary to further add a mechanical mechanism. Alternatively, itis necessary to prepare a large variety of polarization filters havingdifferent polarization orientations, respectively.

Further, according to the technology disclosed in Japanese PatentApplication Laid-open No. H05-048060, one GH liquid crystal layerfunctions as a CF (Color Filter), which filters only one polarizationcomponent. Because of this, even if the GH liquid crystal layerfunctions as a color filter, an unattenuated polarization componentremains. As a result, an ND filter may not be realized, or a deviceconfigured to detect an arbitrary polarization may not be realized.

Further, even if two GH liquid crystal layers are provided and theyfunction as ND filters, a polarization component to be reduced will notbe reduced sufficiently. In addition, the signal level of thepolarization component to be detected simultaneously will be decreased.

In view of the above-mentioned circumstances, it is specificallydesirable to remove a polarization component appropriately even if acamera orientation is changed, and to take an image including apolarization component removed appropriately.

According to an embodiment of the present technology, there is providedan image pickup apparatus, including: a polarization removing elementlayer including a plurality of polarization removing elements, each ofthe plurality of polarization removing elements being configured topolarize an incident light to thereby obtain light having a polarizationaxis in a predetermined direction and to transmit the polarized light,the polarization axis of one polarization removing element intersectingwith the polarization axis of another polarization removing element; apolarizer layer provided in front of the polarization removing elementlayer, the polarizer layer being configured to twist polarization axesof light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light; an inclinationdetecting device configured to detect inclination of the incident lightto an optical axis; a controller configured to control and drive thepolarizer layer such that the polarizer layer twists the polarizationaxis of light depending on the inclination detected by the inclinationdetecting device and transmits the light; and an image pickup deviceconfigured to detect the light passing through the polarization removingelement layer.

The polarizer layer may include a twisted nematic liquid crystal layer.

The controller may be configured to control a twist angle of nematicliquid crystal molecules in the twisted nematic liquid crystal layerbetween about 0 degrees to 270 degrees depending on the inclinationdetected by the inclination detecting device.

The image pickup apparatus may further include matrix wiring configuredto apply voltage to the twisted nematic liquid crystal layer.

The twisted nematic liquid crystal layer may be divided into a pluralityof areas, each of the plurality of areas corresponding to at least onepixel, the wiring being arranged in the matrix form such that the wiringcorresponds to the plurality of areas.

The image pickup apparatus may further include an optimizer processorconfigured to apply subtraction, enhancement, and correction processingto an image detected by the image pickup device. A pixel array mayinclude a pixel of at least one color having a great effect onbrightness, and includes pixels having color combination of at leastthree colors, the controller may be configured to drive the polarizationremoving element layer and the polarizer layer corresponding to a pixelarea of the color having a great effect on brightness, and the optimizerprocessor may be configured to compare activity of each pixel of apolarized image with activity of the corresponding pixel of a normalimage, the polarized image being taken under a state where the polarizerlayer is driven and including a pixel of a color having a great effecton brightness, the normal image being taken and including pixelsexcluding the pixel of the color having a great effect on brightness outof the pixels having color combination of at least three colors, and togenerate a processed image including pixels each having higher activity.

The pixel array may be checkered, the pixel array including two pixelsin a horizontal direction and two pixels in a perpendicular directionfor each unit, a red pixel and a blue pixel being arranged on onediagonal line, a first green pixel and a second green pixel beingarranged on the other diagonal line different from the one diagonalline, the controller may be configured to drive the polarizationremoving element layer and the polarizer layer corresponding to thepixel area of the second green pixel, and the optimizer processor may beconfigured to compare activity of each pixel of a polarized image withactivity of the corresponding pixel of a normal image, the polarizedimage being taken under a state where the polarizer layer is driven andincluding the second green pixel, the normal image being taken andincluding the red pixel, the blue pixel, and the first green pixel, andto generate a processed image including pixels each having higheractivity.

The pixel array may include two pixels in a horizontal direction and twopixels in a perpendicular direction for each unit, the pixel arrayincluding a red pixel, a blue pixel, a green pixel, and a white pixel,the controller may be configured to drive the polarization removingelement layer and the polarizer layer corresponding to the pixel area ofthe white pixel, and the optimizer processor may be configured tocompare activity of each pixel of a polarized image with activity of thecorresponding pixel of a normal image, the polarized image being takenunder a state where the polarizer layer is driven and including thewhite pixel, the normal image being taken and including the red pixel,the blue pixel, and the green pixel, and to generate a processed imageincluding pixels each having higher activity.

The pixel array may be checkered, the pixel array including two pixelsin a horizontal direction and two pixels in a perpendicular directionfor each unit, a red pixel and a blue pixel being arranged on onediagonal line, a first green pixel and a second green pixel beingarranged on the other diagonal line different from the one diagonalline, the controller may be configured to drive the polarizationremoving element layer corresponding to the pixel area of the firstgreen pixel in one direction, and the polarization removing elementlayer corresponding to the pixel area of the second green pixel in adirection orthogonal to the one direction, and to drive the polarizerlayer, and the optimizer processor may be configured to compare activityof each pixel of a first polarized image with activity of thecorresponding pixel of a second polarized image, the first polarizedimage including the first green pixel, the red pixel, and the bluepixel, the second polarized image including the second green pixel, thered pixel, and the blue pixel, and to generate a processed imageincluding pixels each having higher activity.

Each of the polarization removing elements may include a Guest-Hostliquid crystal.

The controller may be configured to drive at least one Guest-Host liquidcrystal out of the plurality of Guest-Host liquid crystals being theplurality of polarization removing elements.

The plurality of polarization removing elements may be two polarizationremoving elements, a polarization axis of one polarization removingelement being orthogonal to a polarization axis of the otherpolarization removing element.

The image pickup apparatus may further include an optimizer processorconfigured to apply subtraction, enhancement, and correction processingto an image detected by the image pickup device.

A transparent layer may be provided on an image pickup device, arefractive index of the transparent layer being lower than a refractiveindex of an on-chip lens, the on-chip lens being structured on a pixelof the image pickup device, the polarizer being arranged in front of thetransparent layer having the lower refractive index.

The polarization removing element may include a wire-grid polarizer.

According to an embodiment of the present technology, there is providedan image pickup method, executed by an image pickup apparatus includinga polarization removing element layer including a plurality ofpolarization removing elements, each of the plurality of polarizationremoving elements being configured to polarize an incident light tothereby obtain light having a polarization axis in a predetermineddirection and to transmit the polarized light, the polarization axis ofone polarization removing element intersecting with the polarizationaxis of another polarization removing element, a polarizer layerprovided in front of the polarization removing element layer, thepolarizer layer being configured to twist polarization axes of lighthaving a plurality of polarization axes by predetermined angles,respectively, and to transmit the light, an inclination detecting deviceconfigured to detect inclination of the incident light to an opticalaxis, a controller configured to control and drive the polarizer layersuch that the polarizer layer twists the polarization axis of lightdepending on the inclination detected by the inclination detectingdevice and transmits the light, and an image pickup device configured todetect the light passing through the polarization removing elementlayer, the method including: detecting, by the inclination detectingdevice, inclination of the incident light to an optical axis;controlling and driving, by the controller, the polarizer layer suchthat the polarizer layer twists the polarization axis of light dependingon the inclination detected by the inclination detecting device andtransmits the light; and detecting, by the image pickup device, thelight passing through the polarization removing element layer.

According to an embodiment of the present technology, there is provideda program, causing a computer configured to control an image pickupapparatus including a polarization removing element layer including aplurality of polarization removing elements, each of the plurality ofpolarization removing elements being configured to polarize an incidentlight to thereby obtain light having a polarization axis in apredetermined direction and to transmit the polarized light, thepolarization axis of one polarization removing element intersecting withthe polarization axis of another polarization removing element, apolarizer layer provided in front of the polarization removing elementlayer, the polarizer layer being configured to twist polarization axesof light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light, an inclinationdetecting device configured to detect inclination of the incident lightto an optical axis, a controller configured to control and drive thepolarizer layer such that the polarizer layer twists the polarizationaxis of light depending on the inclination detected by the inclinationdetecting device and transmits the light, and an image pickup deviceconfigured to detect the light passing through the polarization removingelement layer, to execute the steps of: detecting, by the inclinationdetecting device, inclination of the incident light to an optical axis;controlling and driving, by the controller, the polarizer layer suchthat the polarizer layer twists the polarization axis of light dependingon the inclination detected by the inclination detecting device andtransmits the light; and detecting, by the image pickup device, thelight passing through the polarization removing element layer.

According to an embodiment of the present technology, there is provideda camera system, including: a polarization removing element layerincluding a plurality of polarization removing elements, each of theplurality of polarization removing elements being configured to polarizean incident light to thereby obtain light having a polarization axis ina predetermined direction and to transmit the polarized light, thepolarization axis of one polarization removing element intersecting withthe polarization axis of another polarization removing element; apolarizer layer provided in front of the polarization removing elementlayer, the polarizer layer being configured to twist polarization axesof light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light; an inclinationdetecting device configured to detect inclination of the incident lightto an optical axis; a controller configured to control and drive thepolarizer layer such that the polarizer layer twists the polarizationaxis of light depending on the inclination detected by the inclinationdetecting device and transmits the light; and an image pickup deviceconfigured to detect the light passing through the polarization removingelement layer.

According to an embodiment of the present technology, there is provideda camera main body, including: a polarization removing element layerincluding a plurality of polarization removing elements, each of theplurality of polarization removing elements being configured to polarizean incident light to thereby obtain light having a polarization axis ina predetermined direction and to transmit the polarized light, thepolarization axis of one polarization removing element intersecting withthe polarization axis of another polarization removing element; apolarizer layer provided in front of the polarization removing elementlayer, the polarizer layer being configured to twist polarization axesof light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light; an inclinationdetecting device configured to detect inclination of the incident lightto an optical axis; a controller configured to control and drive thepolarizer layer such that the polarizer layer twists the polarizationaxis of light depending on the inclination detected by the inclinationdetecting device and transmits the light; and an image pickup deviceconfigured to detect the light passing through the polarization removingelement layer.

According to the embodiments of the present technology, an inclinationdetecting device is configured to detect inclination of the incidentlight to an optical axis. A controller is configured to control anddrive the polarizer layer such that the polarizer layer twists thepolarization axis of light depending on the inclination detected by theinclination detecting device and transmits the light. An image pickupdevice is configured to detect the light passing through thepolarization removing element layer.

According to the embodiments of the present technology, it is possibleto remove a polarization component appropriately, and to take an imageincluding a polarization component removed appropriately.

These and other objects, features and advantages of the presentdisclosure will become more apparent in light of the following detaileddescription of best mode embodiments thereof, as illustrated in theaccompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an operation of extracting apolarization component by an image pickup apparatus in the past when acamera orientation is changed;

FIG. 2 is a diagram showing an example of the configuration of a cameraof the present technology;

FIG. 3 is a diagram illustrating an example of the structure of apolarization control device in detail;

FIG. 4 is a flowchart illustrating polarization-component extractionprocessing performed by the camera of FIG. 2;

FIG. 5 is a diagram illustrating the polarization-component extractionprocessing performed by the camera of FIG. 2;

FIG. 6 is a diagram illustrating an image taken in thepolarization-component extraction processing performed by the camera ofFIG. 2;

FIG. 7 is a diagram illustrating an image taken in thepolarization-component extraction processing performed by the camera ofFIG. 2;

FIG. 8 is a diagram illustrating an example of a structure integrallyincluding on-chip lenses and the polarization control device;

FIG. 9 is a diagram illustrating an example of a structure in which thepolarization control device is divided into a plurality of area unitseach including a plurality of pixels of an image pickup device;

FIG. 10 is a diagram illustrating an example of a structure in which thepolarization control device is divided into sets of RGB pixels of theimage pickup device;

FIG. 11 is a diagram illustrating an example of a structure in which thepolarization control device is divided into pixels of the image pickupdevice;

FIG. 12 is a diagram illustrating an example of generating a processedimage having no blown-out highlights and no blocked-up shadows based ona polarized image only including second green pixels, which aresubjected to horizontal-polarization processing, and based on a normalimage including the other pixels;

FIG. 13 is a diagram illustrating an example of generating a processedimage having no blown-out highlights and no blocked-up shadows based ona polarized image only including white pixels, which are subjected tohorizontal-polarization processing, and based on a normal imageincluding the other pixels;

FIG. 14 is a diagram illustrating an example of applying polarizationprocessing to first and second green pixels, and generating a processedimage having no blown-out highlights and no blocked-up shadows based ona first polarized image including red pixels, blue pixels, and firstgreen pixels, and based on a second polarized image including redpixels, blue pixels, and second green pixels;

FIG. 15 is a diagram illustrating an example of providing the colorfilters and the on-chip lenses in front of a TN liquid crystal;

FIG. 16 is a diagram illustrating an example of providing the colorfilters and the on-chip lenses between the TN liquid crystal and the GHliquid crystals;

FIG. 17 is a diagram illustrating an example of using apolarization-removing structural layer;

FIG. 18 is a diagram illustrating types of the polarization-removingstructural layer; and

FIG. 19 is a diagram illustrating an example of the structure of ageneral-purpose personal computer.

DETAILED DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present disclosure will be describedwith reference to the drawings. Note that the embodiments will bedescribed in the following order.

1. First embodiment (example of polarization control device having basicstructure)

2. Second embodiment (example of integrally-provided polarizationcontrol device and on-chip lenses)

3. Third embodiment (example of dividing polarization control deviceinto areas each including plurality of pixels, and controlling thepolarization control device)

4. Fourth embodiment (example of dividing polarization control devicefor each set of RGB pixels, and controlling polarization control device)

5. Fifth embodiment (example of dividing polarization control device foreach pixel, and controlling polarization control device)

6. Sixth embodiment (example of generating processed image based onpolarized image only including second green pixels, which are subjectedto polarization control, and based on normal image including the otherpixels of checkered array of red, green, and blue pixels)

7. Seventh embodiment (example of generating processed image based onpolarized image only including white pixels, which are subjected topolarization control, and based on normal image including the otherpixels of 2 pixels.times.2 pixels array of red, green, blue, and whitepixels)

8. Eighth embodiment (example of applying polarization control to firstand second green pixels of checkered array of red, green, and bluepixels, and generating processed image based on first polarized imageincluding first green, red, and blue pixels and based on secondpolarized image including second green, red, and blue pixels)

9. Ninth embodiment (example of providing on-chip lenses and colorfilters in front of polarization control device)

10. Tenth embodiment (example of providing on-chip lenses and colorfilters between TN liquid crystal and GH liquid crystals of polarizationcontrol device)

11. Eleventh embodiment (example of applying polarization-removingstructural layer to polarization control device)

1. First Embodiment Example of the Structure of a Camera of the PresentTechnology

FIG. 2 shows an example of the structure of a camera of the presenttechnology. The camera 41 of FIG. 2 is configured to detect animage-pickup posture (camera orientation) of a main body, to rotate apolarization axis of a polarization component based on the detectedimage-pickup posture (camera orientation), to appropriately detect lighthaving a polarization component to be detected, and to take an imageincluding light having the appropriately-detected polarizationcomponent.

The camera 41 includes the optical-system block 51, the image pickupdevice 52, the gyro sensor 53, the signal processor 54, the storage 55,the display unit 56, and the operation unit 57.

The optical-system block 51 includes the image pickup lenses 61 and 62,and the polarization control device 63. The image pickup lenses 61 and62 are optical lenses. The image pickup lenses 61 and 62 adjust thefocal position of an object. An incident light passes through the imagepickup lenses 61 and 62, and reaches the polarization control device 63.Note that, in the example of FIG. 2, the two image pickup lenses 61 and62 are provided. Alternatively, one image pickup lens or three or moreimage pickup lenses may be provided. Further, in the example of FIG. 2,the image pickup lenses 61 and 62 are convex lenses. Alternatively,other lenses may be used together. For example, image pickup lenses mayinclude a concave lens.

The polarization control device 63 includes the TN (Twisted Nematic)liquid crystal layer 71, and the GH (Guest-Host) liquid crystal layers72 and 73. The polarization control device 63 is provided in front ofthe image pickup device 52. Further, in the polarization control device63, the ITO (Indium Tin Oxide) electrode layer 81-1 is provided on theTN (Twisted Nematic) liquid crystal layer 71. The ITO (Indium Tin Oxide)electrode layer 81-2 is provided between the TN (Twisted Nematic) liquidcrystal layer 71 and the GH liquid crystal layer 72. The ITO (Indium TinOxide) electrode layer 81-3 is provided between the GH liquid crystallayer 72 and the GH liquid crystal layer 73. The ITO (Indium Tin Oxide)electrode layer 81-4 is provided on the GH liquid crystal layer 73. TheITO (Indium Tin Oxide) electrode layers 81-1 and 81-2 control theoperation of the layer 71. The ITO (Indium Tin Oxide) electrode layers81-2 and 81-3 control the operation of the layer 72. The ITO (Indium TinOxide) electrode layers 81-3 and 81-4 control the operation of the layer73.

Depending on the voltage applied between the ITO electrode layers 81-1and 81-2, the orientation of the molecules of the twisted nematic liquidcrystal of the TN liquid crystal layer 71 is rotated about 90 to 270degrees of the twist angle. The polarization orientation of the entireincident light is rotated depending on the rotation amount of the liquidcrystal molecules. The TN liquid crystal layer 71 is used for a liquidcrystal display apparatus or the like. In this case, polarizers areformed on the both sides of the TN liquid crystal layer 71. The TNliquid crystal layer 71 is sandwiched between the polarizers and isused. However, in this embodiment, the TN liquid crystal layer 71 isused alone.

The GH liquid crystal layer 72 includes liquid crystal molecules (host)and dichroism dye molecules (guest) solved in the liquid crystalmolecules (host). The GH liquid crystal layer 72 is characterized inthat the guest is oriented along the host. Because of this, if a voltageis not applied to the liquid crystal, the liquid crystal molecules(host) are oriented in a plane with respect to an incident light, andthe dye molecules (guest) absorb the polarization component in parallelwith the liquid crystal molecules. If a voltage is applied to the liquidcrystal, the liquid crystal molecules are oriented perpendicularly inthe plane, and the dye molecules (guest) transmit the polarizationcomponent.

More specifically, as shown in the left part of FIG. 3, the electrodes101 and 102 controls the liquid crystal molecules. If a voltage is notapplied between the electrodes 101 and 102, the liquid crystal molecules112-1 to 112-3 are held in the horizontal direction. The dye molecules111-1 to 111-3 are held in the same direction. Because of this, the GHliquid crystal layer 72 absorbs the polarization component, and thepolarization component does not pass through the GH liquid crystal layer72. Meanwhile, as shown in the right part of FIG. 3, the electrodes 101and 102 control the liquid crystal molecules. If a voltage is appliedbetween the electrodes 101 and 102, the liquid crystal molecules 112-1to 112-3 are held in the perpendicular direction. The dye molecules111-1 to 111-3 are held in the same direction. Because of this, thepolarization component passes through the GH liquid crystal layer 72.Note that, here, “dichroism” means the characteristics in which the dyeabsorbs/does not absorb light depending on the traveling direction andthe polarization direction of the light.

The image pickup device 52 is a CCD (Charge Coupled Device) or a CMOS(Complementary Metal-Oxide Semiconductor). Light passes through theoptical-system block 51. The image pickup device 52 photoelectricallyconverts the light into an electric signal. The image pickup device 52supplies the electric signal, as an image signal, to the signalprocessor 54.

The gyro sensor 53 detects the angular speed. The angular speed is usedto determine the camera orientation, i.e., the orientation of the mainbody of the camera 41. The gyro sensor 53 supplies the angular speed tothe signal processor 54.

The signal processor 54 includes the controller 54 a and the optimizer54 b. When a keyboard and operation buttons are operated, the operationunit 57 supplies operation signals to the signal processor 54. Thesignal processor 54 controls the entire behavior of the camera 41 basedon the operation signals. Further, the image pickup device 52 suppliesimage signals to the signal processor 54. The signal processor 54 storesthe image signals in the storage 55. The storage 55 includes a HDD (HardDisc Drive), a SSD (Solid State Drive), and the like. Further, thesignal processor 54 displays the image signals supplied from the imagepickup device 52 on the display unit 56. The display unit 56 includes aLCD (Liquid Crystal Display) and the like.

The gyro sensor 53 supplies information on the angular speed of the mainbody of the camera 41 to the controller 54 a. The controller 54 acalculates the present posture of the camera 41 based on the suppliedinformation. The controller 54 a determines the image-pickup posture(camera orientation), i.e., the direction and posture of the camera.Further, the controller 54 a adjusts the twist angle in the polarizationdirection of the TN liquid crystal layer 71 of the optical-system block51 based on the information on the determined image-pickup posture(camera orientation) of the camera 41. As a result, the controller 54 acontrols the polarization orientation of an object (objective world),whose image is to be taken, such that it is always the same as theorientation of the GH liquid crystal layers 72 and 73, and takes animage.

The optimizer 54 b performs various kinds of processing on image signalssupplied from the image pickup device 52. The various kinds ofprocessing include various kinds of subtraction processing such asforeground or background, enhancement processing such as edgeenhancement, and correction processing such as color correction.

Polarization-Component Detection Processing

Next, with reference to the flowchart of FIG. 4, polarization-componentdetection processing will be described. The camera 41 of FIG. 2 performsthe polarization-component detection processing.

For example, a user operates the operation unit 57, and an instructionto take an image is input. Then the camera 41 starts the processing. InStep S11, in response to the instruction, the gyro sensor 53 detects anangular speed. The angular speed is used to determine the cameraorientation. The camera orientation is the image-pickup posture of themain body of the camera 41. The gyro sensor 53 supplies the angularspeed to the signal processor 54. The controller 54 a of the signalprocessor 54 determines the camera orientation based on the informationon the angular speed.

In Step S12, the controller 54 a calculates, based on the information onthe determined camera orientation, the angle of torsionally rotating thepolarization orientation such that the polarization orientation of thepolarization component of the TN liquid crystal layer 71 corresponds tothe polarization orientation in a case where the camera orientation ishorizontal. The controller 54 a calculates a driving amountcorresponding to the angle.

In Step S13, the controller 54 a applies a voltage, which is required todrive the TN liquid crystal layer 71 by the calculated driving amount,between the ITO electrode layers 81-1 and 81-2. As the result of theprocessing, the TN liquid crystal layer 71 torsionally rotates thepolarization orientation of the polarization component by the anglecorresponding to the camera orientation detected by the gyro sensor 53.Light passes through the TN liquid crystal layer 71, and enters the GHliquid crystal layers 72 and 73.

In Step S14, the controller 54 a applies a predetermined voltage to theITO electrode layers 81-2 to 81-4 to thereby turn on/off each of the GHliquid crystal layers 72 and 73. The light passes through the TN liquidcrystal layer 71, whereby the polarization orientation of thepolarization component of the light is rotated. The GH liquid crystallayer 72 transmits or absorbs light having one polarization orientationout of the light whose polarization component is rotated. The GH liquidcrystal layer 73 transmits or absorbs light having another polarizationorientation out of the light whose polarization component is rotated.

In Step S15, the image pickup device 52 receives and detects the light,which passes through the GH liquid crystal layers 72 and 73. Inaddition, the image pickup device 52 photoelectrically converts thedetected light into an electric signal. As a result, the image pickupdevice 52 generates an image signal as a whole. The image pickup device52 outputs the image signal to the signal processor 54. The controller54 a of the signal processor 54 displays the image signal supplied fromthe image pickup device 52 on the display unit 56. Alternatively, thecontroller 54 a stores the image signal supplied from the image pickupdevice 52 in the storage 55. At this time, the optimizer 54 barbitrarily processes the image signal as necessary.

That is, to summarize the above-mentioned processing, as shown in FIG.5, in the processing of Step S11, the gyro sensor 53 detects the angularspeed. The angular speed is used to determine the camera orientation ofthe camera 41. The gyro sensor 53 supplies the angular speed to thesignal processor 54. For example, the camera 41 is inclined at 45degrees to the horizontal direction. In this case, the gyro sensor 53supplies the angular speed corresponding to 45 degrees to the signalprocessor 54 as a detection result.

Then, in the processing of Step S12, the controller 54 a of the signalprocessor 54 determines the camera orientation. The controller 54 adetermines the angle of rotating the polarization direction of thepolarization component by the angle corresponding to the cameraorientation. The controller 54 a determines the driving amount of the TNliquid crystal layer 71 corresponding to the angle. That is, if thecamera 41 is inclined at 45 degrees, the controller 54 a determines,based on the detection result of the gyro sensor 53, the driving amountof rotating the polarization direction of the polarization component by45 degrees. This driving amount is the driving amount of the TN liquidcrystal layer 71.

In the processing of Step S13, the controller 54 a applies a voltage,which is required to drive the TN liquid crystal layer 71 by thedetermined driving amount, between the ITO electrode layers 81-1 and81-2. In the processing of Step S13, as shown in FIG. 5, thepolarization direction of the image Ph101 is the horizontal direction.The image Ph101 passes through the TN liquid crystal layer 71. As aresult, the polarization direction of the image Ph101 is rotated by 45degrees, whereby the image Ph101′ is obtained. The image Ph101′ entersthe GH liquid crystal layers 72 and 73. Similarly, as shown in FIG. 5,the polarization direction of the image Pv101 is the perpendiculardirection. The image Pv101 passes through the TN liquid crystal layer71. As a result, the polarization direction of the image Pv101 isrotated by 45 degrees, whereby the image Pv101′ is obtained. The imagePv101′ enters the GH liquid crystal layers 72 and 73.

For example, the GH liquid crystal layer 72 transmits or absorbs thepolarization component having the perpendicular polarization direction.The GH liquid crystal 73 transmits or absorbs the polarization componenthaving the horizontal polarization direction. Light having thepolarization component, whose polarization direction is the horizontaldirection, is to be transmitted. In this case, the processing of StepS14 includes the following processing. That is, the controller 54 aapplies no voltage between the ITO electrode layer 81-2 and 81-3 tothereby apply no voltage to the GH liquid crystal layer 72. As a result,the GH liquid crystal layer 72 absorbs the perpendicular polarizationcomponent. In addition, the controller 54 a applies a voltage betweenthe ITO electrode layers 81-3 and 81-4 to thereby apply a voltage to theGH liquid crystal 73. As a result, the horizontal polarization componentpasses through the GH liquid crystal 73.

The above-mentioned processing will be described in detail. Thepolarization direction of the image Ph101 is rotated by 45 degrees,whereby the image Ph101′ is obtained. The polarization component of theimage Ph101′ is rotated such that the polarization direction correspondsto the camera direction. The GH liquid crystal layer 72 transmits orabsorbs the polarization component having the perpendicular polarizationdirection. Even if the GH liquid crystal layer 72 is turned off (i.e.,absorbs light), the image Ph101′ passes through the GH liquid crystallayer 72 because the controlled polarization direction is perpendicularto the GH liquid crystal layer 72. Further, the polarization componentof the image Ph101′ is rotated such that the polarization directioncorresponds to the camera direction. The GH liquid crystal layer 73transmits or absorbs the polarization component having the horizontalpolarization direction. If the GH liquid crystal layer 73 is turned on(i.e., does not absorb light), then the image Ph101′ passes through theGH liquid crystal layer 73. As a result, the GH liquid crystal layers 72and 73 does not absorb the image Ph101′, and the image Ph101″ passesthrough the GH liquid crystal layers 72 and 73.

Meanwhile, the polarization direction of the image Pv101 is rotated by45 degrees, whereby the image Pv101′ is obtained. The polarizationcomponent of the image Pv101′ is rotated such that the polarizationdirection corresponds to the camera direction. The GH liquid crystallayer 72 transmits or absorbs the polarization component having theperpendicular polarization direction. Because the GH liquid crystallayer 72 is turned off (i.e., absorbs light), the GH liquid crystallayer 72 absorbs almost all the image Pv101′. Further, the polarizationcomponent of the image Pv101′ is rotated such that the polarizationdirection corresponds to the camera direction. The GH liquid crystallayer 73 transmits or absorbs the polarization component having thehorizontal polarization direction. If the GH liquid crystal layer 73 isturned on (i.e., does not absorb light), then the image Pv101′ passesthrough the GH liquid crystal layer 73. Even if the GH liquid crystallayer 73 is turned off (i.e., absorbs light), the image Pv101′ passesthrough the GH liquid crystal layer 73 because the controlledpolarization direction is perpendicular to the GH liquid crystal layer73. As a result, the GH liquid crystal 72 absorbs the image Pv101′, andthe almost-black image Pv101″ thus passes through the GH liquid crystallayer 73.

As described above, in the processing of Step S15, the image pickupdevice 52 receives the synthesized image of the images Ph101″ andPv101″. The image pickup device 52 detects the corresponding imagesignal. Note that, in the example of FIG. 5, as described above, theimage Pv101″ is an almost black image. As a result, actually, the imagepickup device 52 detects the image signal having the desiredpolarization component as shown in the image Ph101″.

In this case, the polarization direction of each of the images Ph101″and Pv101″ is rotated by 45 degrees in the direction in which the camera41 is inclined. Because of this, the polarization direction of eachtaken image in the real space is corrected in the horizontal directionand the perpendicular direction. As a result, the images Ph111 and Pv111of FIG. 5 are extracted. The polarization direction of the extractedimages Ph111 and Pv111 is the polarization direction of the real space.Note that each of the images Ph111 and Pv111 illustrated in FIG. 5merely shows the polarization direction represented by the black andwhite stripes.

An image is taken, and an image signal is stored. The image signalincludes information on the polarization component. For example, asshown in the left part of FIG. 6, an image of a window glass is takenoutdoors. In general, the window glass reflects light of the sky and thelike. The image includes such a reflection component. As a result, it isdifficult to see the room interior in the image. However, the right partof FIG. 6 shows a taken image only having the P-polarization component(image from which S-polarization is removed). The reflection componentat the surface of the window glass is removed from the image. Because ofthis, it is possible to obtain information, which may not be obtained inan image taken by means of a general image pickup method. For example,the following technology employing the similar phenomenon is generallyknown. A pair of polarized sunglasses is used to watch the surface ofwater. Then the surface reflectance component at the surface of water isremoved. Visualization of objects under the water is improved.

Further, as shown in the left part of FIG. 7, on a clear and bright day,signs (white line, hazard sign, speed sign, or the like) on a roadsurface may have blown-out highlights because of the surface reflectancecomponent. The same applies to a wet road surface on a rainy day. Thesurface reflectance component is polarization-dependent. The camera 41of FIG. 2 takes an image of a road surface such that the polarizationcomponent being the surface reflectance component is removed. As aresult, as shown in the right part of FIG. 7, it is possible toappropriately take an image of signs (white line, hazard sign, speedsign, or the like) on a road surface.

According to the above-mentioned processing, even if the camera 41 isinclined when taking an image, the polarization direction is rotatedcorresponding to the camera direction, i.e., the direction in which thecamera 41 is inclined. As a result, it is possible to take an imagehaving a polarization component appropriate to the polarizationdirection of the real space.

Further, if an area in an image has blown-out highlights as describedabove, the following technology is generally employed in related art.That is, brightness is decreased sufficiently such that the reflectedarea, which has blown-out highlights, is recognized, and exposure isadjusted. However, if the brightness level is decreased and the exposureis adjusted, a dark area in the area other than the area havingblown-out highlights may have blocked-up shadows. The HDR (high dynamicrange) method employs the above-mentioned common technology. Accordingto the HDR method, a plurality of images are taken. The exposure levelsof the plurality of images are different from each other between theexposure level corresponding to a bright area and the exposure levelcorresponding to a dark area such that the dark area has no blocked-upshadows. After the plurality of images are taken, the plurality ofimages are synthesized by means of signal processing. As a result, theentire area of the obtained image including the darkest area and thebrightest area is recognized.

This method is used as follows. A camera including a solid-state imagepickup device such as a CCD or a CMOS takes an image. All of the sunnysky, the shadow of a building, and the like are in the angle of view ofthe taken image. In this case, there is a large brightness differencebetween the darkest area and the brightest area of the real objects. Inthis case, the obtained dynamic range is not enough because of thesensitivity of the image pickup device 52 and the saturated signalamount. If exposure is adjusted based on a bright area, a dark area hasblocked-up shadows. If exposure is adjusted based on a dark area, abright area has blown-out highlights. In view of this, a plurality ofimages are obtained. The exposure levels of the plurality of images aredifferent from each other. If an area may have blocked-up shadows, theexposure level of the area is adjusted such that the area will not haveblocked-up shadows. If an area may have blown-out highlights, theexposure level of the area is adjusted such that the area will not haveblown-out highlights. The plurality of images are synthesized. As aresult, the entire area of the obtained image including the darkest areaand the brightest area is recognized.

However, it is necessary to synthesize a plurality of images. Thesynthesized images are taken at different time points. As a result, thesynthesized image may be blurred, and the image quality may be lowered.Further, when a moving image is taken, it is necessary to process themoving image in real time. Because of this, it is necessary to takeimages at a frame rate at least more than double, and to synthesize theimages. The processing configuration itself is complicated, and theimage quality may be lowered.

To the contrary, according to the present technology, only onepolarization component is picked up. As a result, it is possible toobtain the following image. That is, the component reflected at thesurface of an object is removed from an obtained image. Further,according to the method of controlling the brightness level of exposure,some brightness information is not removed sufficiently. To thecontrary, according to the present technology, the polarizationcomponent is controlled. As a result, brightness information is removedand obtained. It is thus possible to obtain brightness information,which is not obtained per se.

In view of the above, both the HDR method in the past and the presenttechnology are used. For example, the camera 41 has the function of theoptimizer 54 b. As a result, it is possible to obtain brightnessinformation of objects, which is not obtained in the HDR method in thepast.

Second Embodiment

In the example of the above-mentioned first embodiment, the polarizationcontrol device 63 is provided in front of the image pickup device 52.Alternatively, for example, the polarization control device 63 mayinclude on-chip lenses and color filters integrally. That is, as shownin FIG. 8, the polarization control device 63 further includes theon-chip lenses 91 and the color filters 92. The polarization controldevice 63 has an all-in-one structure including them.

More specifically, as shown in FIG. 8, the on-chip lens 91 and the colorfilter 92 are provided for each pixel between the image pickup device 52and the polarization control device 63. If the refractive index n1 ofthe on-chip lenses 91 is smaller than the refractive index n2 of thespace (transparent layer) 93 between the on-chip lenses 91 and thepolarization control device 63 (i.e., n1<n2), then the on-chip lenses 91are convex lenses as shown in FIG. 8. Meanwhile, if the refractive indexn1 of the on-chip lenses 91 is larger than the refractive index n2(i.e., n1>n2), then the on-chip lenses 91 are concave lenses (notshown).

Third Embodiment

In the above-mentioned second embodiment, the TN liquid crystal layer71, the GH liquid crystal layers 72 and 73, the on-chip lenses 91, andthe color filters 92 are provided on the entire area of the image pickupdevice 52. The TN liquid crystal layer 71 is controlled in onepolarization direction. Alternatively, for example, as shown in FIG. 9,the image pickup device 52 may be divided into a plurality of zones Z1to Z3 (or may be divided into three or more zones). The GH liquidcrystal layer 72 may be turned on/off and the GH liquid crystal layer 73may be turned on/off independently for each zone. That is, in this case,a plurality of sets of the ITO electrode layers 81-1 to 81-4 areprovided in a matrix form. A set of the ITO electrode layers 81-1 to81-4 corresponds to each of the zones Z1 to Z3. With this structure, thezones Z1 to Z3 are controlled independently. The rotation angle of thepolarization direction of the TN liquid crystal layer 71 may be adjustedfor each zone. In addition, the GH liquid crystal layer 72 may be turnedon/off and the GH liquid crystal layer 73 may be turned on/offindependently for each zone.

According to the structure of FIG. 9, when an image is taken, thecontroller 54 a of the signal processor 54 obtains brightnessinformation (i.e., detection result) from the image pickup device 52.The controller 54 a extracts a predetermined polarization component froman area high in brightness. At the same time, the controller 54 a doesnot extract a predetermined polarization component from an area low inbrightness. According to this processing, a predetermined polarizationcomponent is extracted from an area, which may have blown-outhighlights. Examples of such an area include an area including awindshield in a taken image of the left part of FIG. 6 and an areaincluding a roadway in an image of the left part of FIG. 7. An image ofthe other area is taken under the normal condition. As a result, it ispossible to prevent a dark area, which needs no processing, from beingprocessed excessively. It is possible to prevent the amount ofbrightness signals from being reduced unnecessarily. That is, it ispossible to prevent blocked-up shadows from occurring. At the same time,it is possible to take an image including necessary information.

Fourth Embodiment

In the above-mentioned example of FIG. 9, the image pickup device 52 isdivided into the plurality of zones Z1 to Z3. Each of the zones Z1 to Z3has a plurality of pixels. Each zone only extracts a predeterminedpolarization component. Further, as shown in FIG. 10, only apredetermined polarization component may be extracted from each of thepixel units P1 to P9 (or ten or more pixel units, or eight or less pixelunits). Note that a pixel unit includes a set of RGB. In this case, aplurality of sets of the ITO electrode layers 81-1 to 81-4 are providedin a matrix form. A set of the ITO electrode layers 81-1 to 81-4corresponds to each of the RGB pixel units P1 to P9. As a result, eachRGB pixel unit is controlled independently. With this structure, therotation angle of the polarization direction of the TN liquid crystallayer 71 may be adjusted for each RGB pixel unit. In addition, the GHliquid crystal layer 72 may be turned on/off and the GH liquid crystallayer 73 may be turned on/off independently for each RGB pixel unit.

With this structure, it is possible to prevent blown-out highlights andblocked-up shadows from occurring with a higher degree of accuracy. Atthe same time, it is possible to take an image including necessaryinformation. Note that if the size of a pixel is too small in view ofthe thickness of the polarization control device 63, then the accuracyof detecting an oblique-incident light may be decreased. In view ofthis, if the pixel size is increased or the thickness of thepolarization control device 63 is reduced, then it is possible toprevent the detection accuracy from being decreased.

Fifth Embodiment

In the above-mentioned example, it is possible to adjust the rotationangle of the TN liquid crystal layer 71 in the polarization directionfor each of the pixel units P1 to P9, i.e., each RGB pixel unit. Forexample, as shown in FIG. 11, a polarization component may be extractedfor each totally-independent pixel. That is, in the case of FIG. 11, aplurality of sets of the ITO electrode layers 81-1 to 81-4 are providedin a matrix form. A set of the ITO electrode layers 81-1 to 81-4corresponds to each of the pixels Pix1 to Pix3. With this structure, therotation angle of the polarization direction of the TN liquid crystallayer 71 may be adjusted for each pixel. In addition, the GH liquidcrystal layer 72 may be turned on/off and the GH liquid crystal layer 73may be turned on/off independently for each pixel.

With this structure, it is possible to prevent blown-out highlights andblocked-up shadows from occurring with a much higher degree of accuracy.At the same time, it is possible to take an image including necessaryinformation.

Sixth Embodiment

In the above-mentioned example, the rotate of the polarization directionis adjusted for each pixel. An unprecedented image may be taken byapplying this technology.

That is, as shown in the left part of FIG. 12, the red pixel R, thegreen pixels G1 and G2, and the blue pixel B are arrayed checkerwise. Inthis case, the controller 54 a performs horizontal-polarizationprocessing only on the green pixels G2. An image including the redpixels R, the first green pixels G1, and the blue pixels B is takenunder the normal condition. As a result, as shown in the center part ofFIG. 12, the normal image P1 and the polarized image P2 are taken. Thenormal image P1 including the red pixels R, the first green pixels G1,and the blue pixels B is taken under the normal condition. The polarizedimage P2 including the green pixels G2, which is subjected to thehorizontal-polarization processing, is taken.

In this case, the optimizer 54 b compares the activity of each pixel ofthe normal image P1 with the activity of the corresponding pixel of thepolarized image P2. The optimizer 54 b generates the processed image P3.The processed image P3 includes the pixels each having higher activity.Here, the definition of the “pixel having a higher activity” is asfollows. That is, the difference between the pixel value of a certainpixel and the pixel value of an adjacent pixel is large. Alternatively,the sum of the absolute difference of the adjacent pixels in the areaaround a certain pixel is large. The thus generated processed image P3includes a lot of necessary information in the image. Areas low inactivity (e.g., blown-out highlights and blocked-up shadows) are removedfrom the processed image P3 substantially. Further, the normal image P1and the polarized image P2 are taken simultaneously. As a result, theprocessed image P3 is not blurred. As a result, the pixels, which areused to increase the resolution normally, are used effectively. It ispossible to take an unprecedented image while the resolution is notdecreased and the brightness signal is corrected appropriately.

Seventh Embodiment

In the above-mentioned example, the first and second green pixels, thered pixel, and the blue pixel are arrayed checkerwise (2 pixels.times.2pixels). The polarized image P2 is taken by only using the green pixelsG2, which are subjected to the horizontal-polarization processing. Atthe same time, the normal image P1 including the pixels other than thegreen pixels G2, i.e., the red pixels R, the green pixels G1, and theblue pixels B, is taken under the normal condition. The processed imageP3 is generated. The processed image P3 includes the pixels each havinghigher activity. However, a color combination other than theabove-mentioned combination of green, red, and blue pixels may beemployed. The similar effect may be obtained if pixels, whose color hasan effect on brightness, are subjected to the polarization control. Thatis, for example, as shown in the left part of FIG. 13, a white pixel isarrayed instead of the second green pixel. The polarized image P2′ istaken while the polarization of the white pixels is controlled. At thesame time, the normal image P1′ including the red pixels, the bluepixels, and the green pixels is taken. The processed image P3′ isobtained based on the polarized image P2′ and the normal image P1′. As aresult, the similar effect may be obtained.

That is, the controller 54 a performs the horizontal-polarizationprocessing only on the white pixels W. An image including the red pixelsR, the green pixels G, and the blue pixels B is taken under the normalcondition. As shown in the center part of FIG. 13, the normal image P1′and the polarized image P2′ are thus taken. The normal image P1′including the red pixels R, the green pixels G, and the blue pixels B istaken under the normal condition. The polarized image P2′ including thewhite pixels W, which are subjected to the horizontal-polarizationprocessing, is taken.

In this case, the optimizer 54 b compares the activity of each pixel ofthe normal image P1′ with the activity of the corresponding pixel of thepolarized image P2′. The optimizer 54 b generates the processed imageP3′. The processed image P3′ includes the pixels each having higheractivity. The thus generated processed image P3′ includes a lot ofnecessary information in the image. Areas low in activity (e.g.,blown-out highlights and blocked-up shadows) are removed from theprocessed image P3′ substantially. Further, the normal image P1′ and thepolarized image P2′ are taken simultaneously. The processed image P3′ isthus not blurred. As a result, the pixels, which are used to increasethe resolution normally and the brightness, are used effectively. It ispossible to take an unprecedented image while the resolution is notdecreased and the brightness signal is corrected appropriately. Notethat, as described above, the pixels, which are subjected to thepolarization processing to obtain a polarized image, may not be greenpixels or white pixels. Pixels of another color may be subjected to thepolarization processing as long as the color of the pixels has a greateffect on brightness. Further, the color combination may not be thecombination of red, green, and blue pixels, or may not be thecombination of red, green, blue, and white pixels. Alternatively, otherpixel color combination (e.g., ClearVid array, X-Trance array, hexarray, or the like) may be employed. Further, the color combination maynot be the additive combination of three primary colors (red, green, andblue pixels), or may not be the color combination of the red, green,blue, and white pixels. For example, the subtractive combination ofthree primary colors (magenta, cyan, and yellow pixels) may be employedas the color combination.

Eighth Embodiment

In the above-mentioned example of the sixth embodiment, only the secondgreen pixels G2 of the checkered pixel array are subjected to thehorizontal-polarization control. The thus obtained image is thepolarized image P2. At the same time, the normal image P1 including theother pixels (i.e., red pixels R, first green pixels G1, and blue pixelsB) is taken under the normal condition. The processed image P3 isgenerated. The processed image P3 includes the pixels each having higheractivity. In the above-mentioned example of the seventh embodiment, onlythe white pixels W of the checkered pixel array are subjected to thehorizontal-polarization control. The thus obtained image is thepolarized image P2′. At the same time, the normal image P1′ includingthe other pixels (i.e., red pixels R, green pixels G, and blue pixels B)is taken under the normal condition. The processed image P3′ isgenerated. The processed image P3′ includes the pixels each havinghigher activity. However, for example, as shown in FIG. 14, the firstgreen pixels G1 may be subjected to the polarization control in theperpendicular direction. The second green pixels G2 may be subjected tothe polarization control in the horizontal direction. An image includingthe red pixels R and the blue pixels B may be taken under the normalcondition. The polarized image P11 and the polarized image P12 may begenerated. The polarized image P11 includes the red pixels R, the firstgreen pixels G1, and the blue pixels B. The polarized image P11 issubjected to the perpendicular-polarization processing. The polarizedimage P12 includes the red pixels R, the second green pixels G2, and theblue pixels B. The polarized image P12 is subjected to thehorizontal-polarization processing.

That is, the controller 54 a applies the perpendicular-polarizationcontrol to the first green pixels G1. The controller 54 a applies thehorizontal-polarization control to the second green pixels G2. An imageincluding the red pixels R and the blue pixels B is taken under thenormal condition. The first polarized image P11 and the second polarizedimage P12 are taken. The first polarized image P11 includes the redpixels R, the first green pixels G1, and the blue pixels B. The secondpolarized image P12 includes the red pixels R, the second green pixelsG2, and the blue pixels B.

Then, the optimizer 54 b compares the activity of each pixel of thefirst polarized image P11 with the activity of the corresponding pixelof the second polarized image P12. The optimizer 54 b extracts thepixels each having higher activity. The optimizer 54 b obtains theprocessed image P13 including the extracted pixels based on thepolarized images. As a result, it is possible to generate a processedimage based on the first and second polarized images P11 and P12. Thefirst polarized image P11 is subjected to the polarization control inthe perpendicular direction. The second polarized image P12 is subjectedto the polarization control in the horizontal direction. As a result, itis possible to take an unprecedented image while the resolution is notdecreased and the brightness signal is corrected appropriately.

Ninth Embodiment

In the examples of the above-mentioned embodiments, the on-chip lenses91 and the color filters 92 are provided between the image pickup device52 and the polarization control device 63. Any other structure may beemployed as long as the TN liquid crystal layer 71 is provided in frontof the GH liquid crystal layers 72 and 73. For example, as shown in FIG.15, the on-chip lenses 91 and the color filters 92 may be provided infront of the polarization control device 63.

Tenth Embodiment

Similarly, for example, as shown in FIG. 16, the on-chip lenses 91 andthe color filters 92 may be provided between the TN liquid crystal layer71 and the GH liquid crystal layer 72. In this case, the polarizationcontrol device 63 includes the TN liquid crystal layer 71, the GH liquidcrystal layers 72 and 73, the on-chip lenses 91, the color filters 92,and the space (transparent layer) 93 as a whole. Further, the ITOelectrode layers 81-1 and 81-2-1 control the TN liquid crystal layer 71.The ITO electrode layers 81-2-2 and 81-3 control the GH liquid crystallayer 72.

Eleventh Embodiment

In the examples of the above-mentioned embodiments, the GH liquidcrystal layers 72 and 73 are provided to remove polarization components.The GH liquid crystal layer 72 extracts the perpendicular component. TheGH liquid crystal layer 73 extracts the horizontal component.Alternatively, any other structure may be employed as long as it ispossible to remove polarization components. For example, as shown inFIG. 17, the GH liquid crystal layers 72 and 73 are not provided.Instead, the polarization-removing structural layer 171 may be provided.The polarization-removing structural layer 171 is a wire-grid polarizer.

Here, the polarization-removing structural layer 171 is a wire-gridpolarizer. The wire-grid polarizer absorbs and reflects the polarizationcomponent, which is in parallel with the grid direction of thepolarizer. The wire-grid polarizer transmits the perpendicularpolarization component. An image pickup device detects the perpendicularpolarization component. As shown in the left part of FIG. 18, thepolarization-removing structural layer 171 a is an example of thepolarization-removing structural layer 171. The polarization-removingstructural layer 171 a is configured to detect the horizontal andperpendicular polarization components or to determine pixels for each ofa plurality of areas each including a plurality of grid-like pixels orfor each pixel. Further, as shown in the fight part of FIG. 18, thepolarization-removing structural layer 171 b is an example of thepolarization-removing structural layer 171. The polarization-removingstructural layer 171 b is configured to detect the horizontal andperpendicular polarization components or to determine pixels for each ofa plurality of areas each including a plurality of band-like pixels orfor each pixel.

The wire grid of the polarization-removing structural layer 171 isfixed. This is different from the GH liquid crystal layers 72 and 73.Because of this structure, polarization is always detected when taking apicture. Further, the polarization-removing structural layer 171 islikely to be thinner than the GH liquid crystal layers 72 and 73 becauseof the wire grid. As a result, it is possible to reduce influence on anoblique-incident light in a case where the polarization-removingstructural layer 171 is arrayed for each pixel. Note that JapanesePatent Application Laid-open No. 2012-54856 discloses apolarization-removing structural layer that employs a wire-gridpolarizer.

Further, the camera 41 of each of the above-mentioned first embodimentto eleventh embodiment may include the polarization-removing structurallayer 171 instead of the GH liquid crystal layers 72 and 73. Further, inthe above-mentioned embodiments, the GH liquid crystal layers 72 and 73extract orthogonal polarization components. The wire-gridpolarization-removing structural layer 171 extracts orthogonalpolarization components. Alternatively, any structure may be employed aslong as it is possible to detect different polarization components. TheGH liquid crystal layers 72 and 73 may extract two or more differentpolarization components. The wire-grid polarization-removing structurallayer 171 may extract two or more different polarization components.

As described above, according to the present technology, it is possibleto rotate the polarization of the objective world depending on theinclination of the camera 41. It is possible to extract an arbitrarypolarization components when taking an image and when modulating lightand taking an image. Further, in a case where incident angles of keylight on a plane are different from each other, the TN liquid crystallayer is divided into a plurality of areas, and the driving amount ofthe TN liquid crystal layer is controlled independently for each area.As a result, it is possible to correct the polarization rotation amounton a plane. It is possible to maintain the in-plane uniformity under thepolarization state.

Further, a first GH liquid crystal layer removes P-polarization and asecond GH liquid crystal layer removes S-polarization, and vice versa.The two GH liquid crystal layers are driven independently. As a result,it is possible to detect an image having arbitrary polarizationcomponents of the objective world. Further, if both the two GH liquidcrystal layers are driven, they may function as a ND filter whenmodulating light and taking an image. Further, detected component imagesincluding one/two polarization(s) are combined and processedappropriately while blown-out highlights and blocked-up shadows areprevented from occurring. As a result, it is possible to optimize animage, and to obtain an image in which objects are discriminated fromeach other.

Further, the polarization control device is provided on the image pickupdevice. As a result, it is possible to obtain a filter effect for eachpixel or for each area. Further, the two GH liquid crystal layers aredriven simultaneously for each pixel or for each area. As a result, itis possible to reduce light of a partially brighter area (e.g., sky,water surface, light source, etc.).

Meanwhile, the above-mentioned series of processing may be executed byhardware or software. In the case where the series of processing isexecuted by software, a program configuring the software is installed ina computer from a storage medium. For example, the computer is acomputer, which is built in a dedicated hardware. Alternatively, forexample, the computer is a general-purpose personal computer, which iscapable of executing various functions when various programs areinstalled.

FIG. 19 is an example of the configuration of a general-purpose personalcomputer. The personal computer includes a built-in CPU (CentralProcessing Unit) 1001. The CPU 1001 is connected to the input/outputinterface 1005 via the bus 1004. The ROM (Read Only Memory) 1002 and theRAM (Random Access Memory) 1003 are connected to the bus 1004.

The input unit 1006, the output unit 1007, the storage 1008, and thecommunication unit 1009 are connected to the input/output interface1005. The input unit 1006 includes input devices such as a keyboard anda mouse. A user inputs operation commands in the input devices. Theoutput unit 1007 outputs processing operation windows and imagesresulting from processing to a display device. The storage 1008 includesa hard disk drive and the like, which store programs and various kindsof data. The communication unit 1009 includes a LAN (Local Area Network)adaptor and the like. The communication unit 1009 performs communicationprocessing via a network typified by the Internet. Further, the drive1010 is connected to the input/output interface 1005. The drive 1010writes data in the removal medium 1011, and retrieves data from theremoval medium 1011. Examples of the removal medium 1011 include amagnetic disk (including flexible disk), an optical disk (includingCD-ROM (Compact Disc-Read Only Memory) and DVD (Digital VersatileDisc)), a magnetooptical disk (including MD (Mini Disc)), asemiconductor memory, and the like.

The CPU 1001 executes various kinds of processing in response to aprogram stored in the ROM 1002. Further, the CPU 1001 retrieves aprogram from the removal medium 1011 such as a magnetic disk, an opticaldisk, a magnetooptical disk, a semiconductor memory, or the like. TheCPU 1001 installs the retrieved program in the storage 1008. The CPU1001 loads the program in the RAM 1003 from the storage 1008. The CPU1001 executes various kinds of processing in response to the programloaded in the RAM 1003. Further, the RAM 1003 arbitrarily stores datanecessary for the CPU 1001 to execute various kinds of processing, andother data.

In the computer configured as described above, the CPU 1001 loads aprogram stored in, for example, the storage 1008 in the RAM 1003 via theinput/output interface 1005 and the bus 1004, and executes the program.The CPU 1001 thus executes the above-mentioned series of processing.

For example, the program executed by the computer (the CPU 1001) may bestored in the removal medium 1011, and provided as a packaged medium orthe like. Further, the program may be provided via a wired/wirelesstransmission medium such as a local area network, the Internet, or adigital satellite broadcasting.

The removal medium 1011 may be inserted in the drive 1010 of thecomputer, and the program may thus be installed in the storage 1008 viathe input/output interface 1005. Further, the communication unit 1009may receive the program via a wired/wireless transmission medium, andthe program may be installed in the storage 1008. Alternatively, theprogram may be installed in the ROM 1002 or the storage 1008 previously.

Note that the computer may execute processing in time series in theorder described in the specification in response to a program.Alternatively, the computer may execute processing in parallel inresponse to a program. Alternatively, the computer may executeprocessing at necessary timing (e.g., when program is called) inresponse to a program.

Further, the embodiments of the present technology are not limited tothe above-mentioned embodiments. The embodiments of the presenttechnology may be variously modified within the scope of the presenttechnology.

For example, the present technology may be configured as cloudcomputing. In the cloud computing, a plurality of apparatuses share andcooperatively process one function via a network.

Further, one apparatus may execute the steps described with reference tothe above-mentioned flowchart. Alternatively, a plurality of apparatusesmay share and execute the steps.

Further, if one step includes a plurality of kinds of processing, oneapparatus may execute the plurality of kinds of processing in the step.Alternatively, a plurality of apparatuses may share and execute theprocessing.

Further, the present technology may employ the following structures.

(1) An image pickup apparatus, comprising:

a polarization removing element layer including a plurality ofpolarization removing elements, each of the plurality of polarizationremoving elements being configured to polarize an incident light tothereby obtain light having a polarization axis in a predetermineddirection and to transmit the polarized light, the polarization axis ofone polarization removing element intersecting with the polarizationaxis of another polarization removing element;

a polarizer layer provided in front of the polarization removing elementlayer, the polarizer layer being configured to twist polarization axesof light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light;

an inclination detecting device configured to detect inclination of theincident light to an optical axis;

a controller configured to control and drive the polarizer layer suchthat the polarizer layer twists the polarization axis of light dependingon the inclination detected by the inclination detecting device andtransmits the light; and

an image pickup device configured to detect the light passing throughthe polarization removing element layer.

(2) The image pickup apparatus according to (1), wherein

the polarizer layer includes a twisted nematic liquid crystal layer.

(3) The image pickup apparatus according to (1), wherein

the controller is configured to control a twist angle of nematic liquidcrystal molecules in the twisted nematic liquid crystal layer betweenabout 0 degrees to 270 degrees depending on the inclination detected bythe inclination detecting device.

(4) The image pickup apparatus according to (3), further comprising:

matrix wiring configured to apply voltage to the twisted nematic liquidcrystal layer.

(5) The image pickup apparatus according to (4), wherein

the twisted nematic liquid crystal layer is divided into a plurality ofareas, each of the plurality of areas corresponding to at least onepixel, the wiring being arranged in the matrix form such that the wiringcorresponds to the plurality of areas.

(6) The image pickup apparatus according to (5), further comprising:

an optimizer processor configured to apply subtraction, enhancement, andcorrection processing to an image detected by the image pickup device,wherein

a pixel array includes a pixel of at least one color having a greateffect on brightness, and includes pixels having color combination of atleast three colors,

the controller is configured to drive the polarization removing elementlayer and the polarizer layer corresponding to a pixel area of the colorhaving a great effect on brightness, and

the optimizer processor is configured

to compare activity of each pixel of a polarized image with activity ofthe corresponding pixel of a normal image, the polarized image beingtaken under a state where the polarizer layer is driven and including apixel of a color having a great effect on brightness, the normal imagebeing taken and including pixels excluding the pixel of the color havinga great effect on brightness out of the pixels having color combinationof at least three colors, and

to generate a processed image including pixels each having higheractivity.

(7) The image pickup apparatus according to (6), wherein

the pixel array is checkered, the pixel array including two pixels in ahorizontal direction and two pixels in a perpendicular direction foreach unit, a red pixel and a blue pixel being arranged on one diagonalline, a first green pixel and a second green pixel being arranged on theother diagonal line different from the one diagonal line,

the controller is configured to drive the polarization removing elementlayer and the polarizer layer corresponding to the pixel area of thesecond green pixel, and

the optimizer processor is configured

to compare activity of each pixel of a polarized image with activity ofthe corresponding pixel of a normal image, the polarized image beingtaken under a state where the polarizer layer is driven and includingthe second green pixel, the normal image being taken and including thered pixel, the blue pixel, and the first green pixel, and

to generate a processed image including pixels each having higheractivity.

(8) The image pickup apparatus according to (6), wherein

the pixel array includes two pixels in a horizontal direction and twopixels in a perpendicular direction for each unit, the pixel arrayincluding a red pixel, a blue pixel, a green pixel, and a white pixel,

the controller is configured to drive the polarization removing elementlayer and the polarizer layer corresponding to the pixel area of thewhite pixel, and

the optimizer processor is configured

to compare activity of each pixel of a polarized image with activity ofthe corresponding pixel of a normal image, the polarized image beingtaken under a state where the polarizer layer is driven and includingthe white pixel, the normal image being taken and including the redpixel, the blue pixel, and the green pixel, and

to generate a processed image including pixels each having higheractivity.

(9) The image pickup apparatus according to (6), wherein

the pixel array is checkered, the pixel array including two pixels in ahorizontal direction and two pixels in a perpendicular direction foreach unit, a red pixel and a blue pixel being arranged on one diagonalline, a first green pixel and a second green pixel being arranged on theother diagonal line different from the one diagonal line,

the controller is configured

to drive the polarization removing element layer corresponding to thepixel area of the first green pixel in one direction, and thepolarization removing element layer corresponding to the pixel area ofthe second green pixel in a direction orthogonal to the one direction,and

to drive the polarizer layer, and

the optimizer processor is configured

to compare activity of each pixel of a first polarized image withactivity of the corresponding pixel of a second polarized image, thefirst polarized image including the first green pixel, the red pixel,and the blue pixel, the second polarized image including the secondgreen pixel, the red pixel, and the blue pixel, and

to generate a processed image including pixels each having higheractivity.

(10) The image pickup apparatus according to (1), wherein

each of the polarization removing elements includes a Guest-Host liquidcrystal.

(11) The image pickup apparatus according to (10), wherein

the controller is configured to drive at least one Guest-Host liquidcrystal out of the plurality of Guest-Host liquid crystals being theplurality of polarization removing elements.

(12) The image pickup apparatus according to (1), wherein

the plurality of polarization removing elements are two polarizationremoving elements, a polarization axis of one polarization removingelement being orthogonal to a polarization axis of the otherpolarization removing element.

(13) The image pickup apparatus according to (1), further comprising:

an optimizer processor configured to apply subtraction, enhancement, andcorrection processing to an image detected by the image pickup device.

(14) The image pickup apparatus according to (1), wherein

a transparent layer is provided on an image pickup device, a refractiveindex of the transparent layer being lower than a refractive index of anon-chip lens, the on-chip lens being structured on a pixel of the imagepickup device, the polarizer being arranged in front of the transparentlayer having the lower refractive index.

(15) The image pickup apparatus according to (1), wherein

the polarization removing element includes a wire-grid polarizer.

(16) An image pickup method, executed by an image pickup apparatusincluding

a polarization removing element layer including a plurality ofpolarization removing elements, each of the plurality of polarizationremoving elements being configured to polarize an incident light tothereby obtain light having a polarization axis in a predetermineddirection and to transmit the polarized light, the polarization axis ofone polarization removing element intersecting with the polarizationaxis of another polarization removing element,

a polarizer layer provided in front of the polarization removing elementlayer, the polarizer layer being configured to twist polarization axesof light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light,

an inclination detecting device configured to detect inclination of theincident light to an optical axis,

a controller configured to control and drive the polarizer layer suchthat the polarizer layer twists the polarization axis of light dependingon the inclination detected by the inclination detecting device andtransmits the light, and

an image pickup device configured to detect the light passing throughthe polarization removing element layer,

the method comprising:

detecting, by the inclination detecting device, inclination of theincident light to an optical axis;

controlling and driving, by the controller, the polarizer layer suchthat the polarizer layer twists the polarization axis of light dependingon the inclination detected by the inclination detecting device andtransmits the light; and

detecting, by the image pickup device, the light passing through thepolarization removing element layer.

(17) A program, causing a computer configured to control an image pickupapparatus including

a polarization removing element layer including a plurality ofpolarization removing elements, each of the plurality of polarizationremoving elements being configured to polarize an incident light tothereby obtain light having a polarization axis in a predetermineddirection and to transmit the polarized light, the polarization axis ofone polarization removing element intersecting with the polarizationaxis of another polarization removing element,

a polarizer layer provided in front of the polarization removing elementlayer, the polarizer layer being configured to twist polarization axesof light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light,

an inclination detecting device configured to detect inclination of theincident light to an optical axis,

a controller configured to control and drive the polarizer layer suchthat the polarizer layer twists the polarization axis of light dependingon the inclination detected by the inclination detecting device andtransmits the light, and

an image pickup device configured to detect the light passing throughthe polarization removing element layer,

to execute the steps of:

detecting, by the inclination detecting device, inclination of theincident light to an optical axis;

controlling and driving, by the controller, the polarizer layer suchthat the polarizer layer twists the polarization axis of light dependingon the inclination detected by the inclination detecting device andtransmits the light; and

detecting, by the image pickup device, the light passing through thepolarization removing element layer.

(18) A camera system, comprising:

a polarization removing element layer including a plurality ofpolarization removing elements, each of the plurality of polarizationremoving elements being configured to polarize an incident light tothereby obtain light having a polarization axis in a predetermineddirection and to transmit the polarized light, the polarization axis ofone polarization removing element intersecting with the polarizationaxis of another polarization removing element;

a polarizer layer provided in front of the polarization removing elementlayer, the polarizer layer being configured to twist polarization axesof light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light;

an inclination detecting device configured to detect inclination of theincident light to an optical axis;

a controller configured to control and drive the polarizer layer suchthat the polarizer layer twists the polarization axis of light dependingon the inclination detected by the inclination detecting device andtransmits the light; and

an image pickup device configured to detect the light passing throughthe polarization removing element layer.

(19) A camera main body, comprising:

a polarization removing element layer including a plurality ofpolarization removing elements, each of the plurality of polarizationremoving elements being configured to polarize an incident light tothereby obtain light having a polarization axis in a predetermineddirection and to transmit the polarized light, the polarization axis ofone polarization removing element intersecting with the polarizationaxis of another polarization removing element;

a polarizer layer provided in front of the polarization removing elementlayer, the polarizer layer being configured to twist polarization axesof light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light;

an inclination detecting device configured to detect inclination of theincident light to an optical axis;

a controller configured to control and drive the polarizer layer suchthat the polarizer layer twists the polarization axis of light dependingon the inclination detected by the inclination detecting device andtransmits the light; and

an image pickup device configured to detect the light passing throughthe polarization removing element layer.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

The invention claimed is:
 1. An image pickup apparatus, comprising: apolarizer layer configured to twist polarization axes of light having aplurality of polarization axes by predetermined angles, respectively,and to transmit the light; an inclination detecting device configured todetect inclination of the incident light to an optical axis; acontroller configured to control and drive the polarizer layer such thatthe polarizer layer twists the polarization axis of light depending onthe inclination detected by the inclination detecting device andtransmits the light; and an image pickup device configured to detect thelight passing through the polarizer layer.
 2. The image pickup apparatusaccording to claim 1, wherein the polarizer layer includes a twistednematic liquid crystal layer.
 3. The image pickup apparatus according toclaim 1, wherein the controller is configured to control a twist angleof nematic liquid crystal molecules in the twisted nematic liquidcrystal layer between about 0 degrees to 270 degrees depending on theinclination detected by the inclination detecting device.
 4. The imagepickup apparatus according to claim 3, further comprising: matrix wiringconfigured to apply voltage to the twisted nematic liquid crystal layer.5. The image pickup apparatus according to claim 4, wherein the twistednematic liquid crystal layer is divided into a plurality of areas, eachof the plurality of areas corresponding to at least one pixel, thewiring being arranged in the matrix form such that the wiringcorresponds to the plurality of areas.
 6. The image pickup apparatusaccording to claim 5, further comprising: a polarization removingelement layer; and an optimizer processor configured to applysubtraction, enhancement, and correction processing to an image detectedby the image pickup device, wherein a pixel array includes a pixel of atleast one color having a great effect on brightness, and includes pixelshaving color combination of at least three colors, the controller isconfigured to drive the polarization removing element layer and thepolarizer layer corresponding to a pixel area of the color having agreat effect on brightness, and the optimizer processor is configured tocompare activity of each pixel of a polarized image with activity of thecorresponding pixel of a normal image, the polarized image being takenunder a state where the polarizer layer is driven and including a pixelof a color having a great effect on brightness, the normal image beingtaken and including pixels excluding the pixel of the color having agreat effect on brightness out of the pixels having color combination ofat least three colors, and to generate a processed image includingpixels each having higher activity.
 7. The image pickup apparatusaccording to claim 6, wherein the pixel array is checkered, the pixelarray including two pixels in a horizontal direction and two pixels in aperpendicular direction for each unit, a red pixel and a blue pixelbeing arranged on one diagonal line, a first green pixel and a secondgreen pixel being arranged on the other diagonal line different from theone diagonal line, the controller is configured to drive thepolarization removing element layer and the polarizer layercorresponding to the pixel area of the second green pixel, and theoptimizer processor is configured to compare activity of each pixel of apolarized image with activity of the corresponding pixel of a normalimage, the polarized image being taken under a state where the polarizerlayer is driven and including the second green pixel, the normal imagebeing taken and including the red pixel, the blue pixel, and the firstgreen pixel, and to generate a processed image including pixels eachhaving higher activity.
 8. The image pickup apparatus according to claim6, wherein the pixel array includes two pixels in a horizontal directionand two pixels in a perpendicular direction for each unit, the pixelarray including a red pixel, a blue pixel, a green pixel, and a whitepixel, the controller is configured to drive the polarization removingelement layer and the polarizer layer corresponding to the pixel area ofthe white pixel, and the optimizer processor is configured to compareactivity of each pixel of a polarized image with activity of thecorresponding pixel of a normal image, the polarized image being takenunder a state where the polarizer layer is driven and including thewhite pixel, the normal image being taken and including the red pixel,the blue pixel, and the green pixel, and to generate a processed imageincluding pixels each having higher activity.
 9. The image pickupapparatus according to claim 6, wherein the pixel array is checkered,the pixel array including two pixels in a horizontal direction and twopixels in a perpendicular direction for each unit, a red pixel and ablue pixel being arranged on one diagonal line, a first green pixel anda second green pixel being arranged on the other diagonal line differentfrom the one diagonal line, the controller is configured to drive thepolarization removing element layer corresponding to the pixel area ofthe first green pixel in one direction, and the polarization removingelement layer corresponding to the pixel area of the second green pixelin a direction orthogonal to the one direction, and to drive thepolarizer layer, and the optimizer processor is configured to compareactivity of each pixel of a first polarized image with activity of thecorresponding pixel of a second polarized image, the first polarizedimage including the first green pixel, the red pixel, and the bluepixel, the second polarized image including the second green pixel, thered pixel, and the blue pixel, and to generate a processed imageincluding pixels each having higher activity.
 10. The image pickupapparatus according to claim 1, further comprising: a polarizationremoving element layer including a plurality of polarization removingelements, wherein each of the polarization removing elements includes aGuest-Host liquid crystal.
 11. The image pickup apparatus according toclaim 10, wherein the controller is configured to drive at least oneGuest-Host liquid crystal out of the plurality of Guest-Host liquidcrystals being the plurality of polarization removing elements.
 12. Theimage pickup apparatus according to claim 1, further comprising: apolarization removing element layer including two polarization removingelements, a polarization axis of one polarization removing element beingorthogonal to a polarization axis of the other polarization removingelement.
 13. The image pickup apparatus according to claim 1, furthercomprising: an optimizer processor configured to apply subtraction,enhancement, and correction processing to an image detected by the imagepickup device.
 14. The image pickup apparatus according to claim 1,wherein a transparent layer is provided on an image pickup device, arefractive index of the transparent layer being lower than a refractiveindex of an on-chip lens, the on-chip lens being structured on a pixelof the image pickup device, the polarizer being arranged in front of thetransparent layer having the lower refractive index.
 15. The imagepickup apparatus according to claim 1, wherein the polarizer layerincludes a wire-grid polarizer.
 16. An image pickup method, executed byan image pickup apparatus including a polarizer layer configured totwist polarization axes of light having a plurality of polarization axesby predetermined angles, respectively, and to transmit the light, aninclination detecting device configured to detect inclination of theincident light to an optical axis, a controller configured to controland drive the polarizer layer such that the polarizer layer twists thepolarization axis of light depending on the inclination detected by theinclination detecting device and transmits the light, and an imagepickup device configured to detect the light passing through thepolarization removing element layer, the method comprising: detecting,by the inclination detecting device, inclination of the incident lightto an optical axis; controlling and driving, by the controller, thepolarizer layer such that the polarizer layer twists the polarizationaxis of light depending on the inclination detected by the inclinationdetecting device and transmits the light; and detecting, by the imagepickup device, the light passing through the polarizer layer.
 17. Anon-transitory computer readable medium containing a program that, whenexecuted, causes a computer configured to control an image pickupapparatus including a polarizer layer configured to twist polarizationaxes of light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light, an inclinationdetecting device configured to detect inclination of the incident lightto an optical axis, a controller configured to control and drive thepolarizer layer such that the polarizer layer twists the polarizationaxis of light depending on the inclination detected by the inclinationdetecting device and transmits the light, and an image pickup deviceconfigured to detect the light passing through the polarization removingelement layer, to execute the operations of: detecting, by theinclination detecting device, inclination of the incident light to anoptical axis; controlling and driving, by the controller, the polarizerlayer such that the polarizer layer twists the polarization axis oflight depending on the inclination detected by the inclination detectingdevice and transmits the light; and detecting, by the image pickupdevice, the light passing through the polarizer layer.
 18. A camerasystem, comprising: a polarizer layer configured to twist polarizationaxes of light having a plurality of polarization axes by predeterminedangles, respectively, and to transmit the light; an inclinationdetecting device configured to detect inclination of the incident lightto an optical axis; a controller configured to control and drive thepolarizer layer such that the polarizer layer twists the polarizationaxis of light depending on the inclination detected by the inclinationdetecting device and transmits the light; and an image pickup deviceconfigured to detect the light passing through the polarizer layer. 19.A camera main body, comprising: a polarizer layer configured to twistpolarization axes of light having a plurality of polarization axes bypredetermined angles, respectively, and to transmit the light; aninclination detecting device configured to detect inclination of theincident light to an optical axis; a controller configured to controland drive the polarizer layer such that the polarizer layer twists thepolarization axis of light depending on the inclination detected by theinclination detecting device and transmits the light; and an imagepickup device configured to detect the light passing through thepolarizer layer.
 20. The image pickup apparatus according to claim 1,further comprising a polarization removing element layer including aplurality of polarization removing elements, each of the plurality ofpolarization removing elements being configured to polarize an incidentlight to thereby obtain light having a polarization axis in apredetermined direction and to transmit the polarized light, thepolarization axis of one polarization removing element intersecting withthe polarization axis of another polarization removing element.
 21. Theimage pickup apparatus according to claim 1, further comprising: a pixelarray including a unit pixel having two subpixels in a horizontaldirection and two subpixels in a vertical direction, wherein respectiveones of the subpixels have a corresponding color filter and acorresponding polarizer sublayer of the polarizer layer, each colorfilter corresponds to the same color, and each polarizer sublayercorresponds to a different polarization axis.
 22. The image pickupmethod according to claim 16, wherein the image pickup apparatus furtherincludes a polarization removing element layer including a plurality ofpolarization removing elements, each of the plurality of polarizationremoving elements being configured to polarize an incident light tothereby obtain light having a polarization axis in a predetermineddirection and to transmit the polarized light, the polarization axis ofone polarization removing element intersecting with the polarizationaxis of another polarization removing element.
 23. The non-transitorycomputer readable medium according to claim 17, wherein the image pickupapparatus further includes a polarization removing element layerincluding a plurality of polarization removing elements, each of theplurality of polarization removing elements being configured to polarizean incident light to thereby obtain light having a polarization axis ina predetermined direction and to transmit the polarized light, thepolarization axis of one polarization removing element intersecting withthe polarization axis of another polarization removing element.
 24. Thecamera system according to claim 18, further comprising a polarizationremoving element layer including a plurality of polarization removingelements, each of the plurality of polarization removing elements beingconfigured to polarize an incident light to thereby obtain light havinga polarization axis in a predetermined direction and to transmit thepolarized light, the polarization axis of one polarization removingelement intersecting with the polarization axis of another polarizationremoving element.
 25. The camera main body according to claim 19,further comprising a polarization removing element layer including aplurality of polarization removing elements, each of the plurality ofpolarization removing elements being configured to polarize an incidentlight to thereby obtain light having a polarization axis in apredetermined direction and to transmit the polarized light, thepolarization axis of one polarization removing element intersecting withthe polarization axis of another polarization removing element.